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(54) System for coding and displaying a plurality of pictures 



(57) A method and system for displaying a plurality 
of specified ones of pictures in an array of windows pro- 
vided on a display screen of a computer based system. 
The size of moving and/or still pictures is reduced into 
size-reduced pictures with a size of the frames. The 
size-reduced pictures are coded into coded size- 
reduced pictures and stored in a hard disc. In response 
to a predetermined signal and a list of pictures to be dis- 
played, a display frame is compiled from the coded size- 
reduced pictures specified by said list. The compiled 
frame is decoded and displayed. 
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Description 

BACKGROUND OF THE INVENTION 
5 1 . Field of the Invention 

[0001] The invention relates to a method and system for displaying an array of reduced-size pictures in a frame and 
to apparatuses incorporating the same such as a video editing apparatus, a movie or image data base system, a video 
monitoring apparatus, etc. The displayed pictures may be still pictures, frames of different videos and/or different frames 
10 of a single video (A video means a series of moving pictures). 

2. Description of the Prior Art 

[0002] There are a variety of systems that need to display a plurality of pictures at a time. Japanese unexamined pat- 
15 ent publication No. Hei5- 19731 issued January 29, 1993 discloses a moving picture image display device. In this 
device, each of the video is encoded in a predetermined format and stored in a hard disc in advance. Displaying a plu- 
rality of moving pictures at a time is achieved by decoding only intra-coded frames of desired moving, reducing the size 
of the decoded frames into an appropriate size, and displaying the reduced-size frames at a time as a single moving 
picture. The intra-coded frames used for display are frames which have been coded as a still picture by means of DCT 
20 (discrete cosine transform) and accordingly can be decoded alone without a need of referring to other frame(s). 

[0003] However, since each of the frames to be displayed has to be decoded in this scheme, if a larger number of 
frames are to be displayed at a time, the scheme requires more decoders or a faster decoder. Also, it takes a significant 
time to display a plurality of pictures because the operations of reduction and decoding are performed each time of dis- 
play. 

25 [0004] It is therefore an object of the invention to provide a system and method for displaying an array of desired pic- 
tures from available moving and/or still pictures at a time with a single decoder by reducing the size of the available mov- 
ing and/or still pictures before encoding them. 

SUMMARY OF THE INVENTION 

30 

[0005] According to the invention, a method of displaying a plurality of specified ones of pictures in an array of win- 
dows provided on a display screen of a computer based system is provided. The method comprises the steps of reduc- 
ing in size the pictures into size-reduced pictures with a size of the frames; coding the size-reduced pictures into coded 
size-reduced pictures; storing the coded size-reduced pictures in a mass storage device; in response to a predeter- 
35 mined signal and a list of pictures to be displayed, compiling a frame comprising coded size-reduced pictures specified 
by the list; decoding the compiled frame for display. 

BRIEF DESCRIPTION OF THE DRAWING 

40 [0006] Further objects and advantages of the present invention will be apparent from the following description of the 
preferred embodiments of the invention as illustrated in the accompanying drawing, in which: 

FIG. 1 is a flow chart showing a basic concept of a method for displaying a plurality of videos or moving pictures at 
a tune in accordance with the principles of the invention; 
45 FIG. 2 is a schematic block diagram showing a client-server video (moving picture) data base system that incorpo- 

rates an illustrative embodiment of the invention; 

FIG. 3 is a diagram showing exemplary contents of the hard disc of FIG 2; 

FIG. 4 is a diagram showing how a coding operation causes the fixed-length slices of a size-reduced video to 
become variable in length; 

so FIG. 5 is a diagram showing the way in which a coded size-reduced video is stored in the hard disc 104; 

FIG. 6 is a diagram showing how desired size-reduced pictures are displayed in an array of windows; 
FIG. 7 is a diagram showing an exemplary output video map in case of the size-reduced videos to be displayed 
being arranged as shown in FIG. 6; 

FIG. 8 is a diagram showing another exemplary output video map in case of the size-reduced videos to be dis- 
ss played being arranged sequentially in windows (1. 1), (2, 1) (X, 1), (1,2) and so on; 

FIG. 9 is a diagram showing a temporal correlation between the compiled frames Fk (k = 1, 2,...) and the frames 
Fj j (j = 1 , 2,...) of a video with a higher frame rate Ri; 

FIG. 10 is a diagram showing a temporal correlation between the compiled frames Fk (k = 1 , 2,...) and the frames 
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Fj j (j = 1 , 2,...) of a video with a lower frame rate Ri; 

FIG. 1 1 is a flow chart showing an exemplary operation of displaying a plurality of videos of various frame rates at 
a time in accordance with the principles of the invention. 

FIG. 1 2 is a diagram showing an exemplary operation of displaying a plurality of videos including a video to be dis- 
5 played only for frame numbers 100 through 299; 

FIG. 13 is a diagram showing an exemplary structure of a preferable output video; 

FIG. 14 is a flow chart showing an exemplary operation in which the frame compiler logic 109 compiles a display 

frame Fk from the selected frames F 1 j, F 2 j F V j in response to the direction from the controller 1 13; 

FIG. 1 5 is a schematic block diagram showing a video decoder used in a video DB server in accordance with a sec- 
w ond embodiment of the invention; 

FIG. 16 is an example of a slice information table 180 generated by the slice INFO generator 169 at the time of 
encoding the size-reduced video Pi; 

FIG. 1 7 is a diagram showing an example of a multiplexed coded size-reduced video data stream generated by the 
video encoder of FIG. 15; 

is FIG. 18 is a diagram showing how the multiplexed coded size-reduced video data is distributed to and stored in a 

plurality of hard discs; 

FIG. 19 is a schematic block diagram showing a stand-alone system that incorporates an illustrative embodiment 
of the invention; and 

FIG. 20 is a schematic block diagram showing a remote monitoring system 3 that incorporates a third illustrative 
20 embodiment of the invention. 

[0007] Throughout the drawing, the same elements when shown in more than one figure are designated by the same 
reference numerals. 

25 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0008] FIG. 2 is a schematic block diagram showing a client-server video (moving picture) data base (DB) system 1 
that incorporates an illustrative embodiment of the invention. The video data base system comprises: 

30 a video database server 10 which collects and manages a variety of videos; 

a plurality of client terminals 20 which permit the users to retrieve desired videos from the video data base server 
10; and 

a transmission network 30 through which the video data abase server 1 0 and the client terminals 20 communicates 
with each other. 

35 

[0009] The transmission network 30 may be a LAN (local-area network) comprising a high-speed data transmission 
system such as an Ethernet (the IEEE 802.3 standard), a token-ring network, etc or a WAN (wide-area network) that 
enables communications in a relatively large geographical area through telephone lines or radio waves. 
[0010] In order to take in video data, the video DB server 10 at least comprises either a video interface (IF) 101 for 
40 taking in video pixel data or a combination of a video taking-in device 1 02 such as an optical disc drive for replaying an 
optical disc to taking in coded video data and a video decoder 1 03 for decoding the taken-in coded video data into video 
pixel data. It is assumed that the server 10 has both of them. The server 10 further comprises: 

a hard disc 104 for storing various programs and data including coded videos; 
45 a picture size reducing logic 105 for reducing the size of the video pixel data from the video decoder 103 or the 
video interface 101 into an appropriate size, i.e., an X-th in width and a Y-th in height, respectively, where M and N 
are appropriate natural numbers; 

a video encoder 107 for encoding the size-reduced video data from the picture size reducing logic 105 into a coded 
size-reduced video for display, which is then stored in the hard disc 103, and for encoding-the video pixel data from 
so the video IF 101 into a full size coded video for offering as actual video data in response to a final or formal request 
from a client 20, which video is then stored in the hard disc 104; 

a frame compiler 1 09 for compiling a frame comprising an array of coded videos from desired ones of the coded 
size-reduced videos stored in the hard disc 104; 

a communication interface 111 for receiving service requests from the clients 20 and for transmitting the compiled 
55 frame of coded size-reduced videos and a formally requested full-size coded video to the clients 20 through the 
transmission network 30; 

a controller 113 for controlling the whole video DB server 10; and 
a system bus 1 17 for interconnecting the above mentioned elements. 



3 



BNSDOCID: <EP 0915622A2. I_> 



EP 0 915 622 A2 



The controller 113 comprises a not-shown CPU (central processing unit), a not-shown read only memory and RAM 
(random access memory) 1 15 as is well known in the art. It is noted that if the server 10 is provided with the video IF 
101 , then the video encoder 107 is so arranged as to be available for both of a size-reduced video and a full size video. 
[001 1] The client terminal 20 comprises: 

5 

a communication interface (IF) 121 for communicating with the video DB server 10; 

a video decoder 123 for decoding a received frame of coded size-reduced videos from the communication IF 121 
into a frame of digital video data; 

a digital-to-analog converter (DAC) 125 for converting the frame of digital video data into a frame of analog video 
io data; 

an NTSC/PAL (National Television Standard Committee/Phase Alternation by Line) encoder 127 for encoding the 
frame of analog video data into an NTSC or PAL format; 
a controller 135 for controlling the operation of the video decoder 123; 
a user interface 129 for providing the controller 135 with an interface with the user; 
is a display device 131 connected with the user IF 129 for displaying the frame of size-reduced videos; and 

input devices 133 such as a keyboard, a mouse, etc. The controller 135 comprises a not-shown CPU (central 
processing unit), a not-shown read only memory and a not-shown random access memory as is well known in the 
art. 

20 [001 2] FIG. 3 shows exemplary contents of the hard disc 1 04 of FIG. 2. In FIG. 3, single line boxes indicate programs, 
and shaded boxes indicate data. The hard disc 104 stores: 

an operating system 141 ; 

taking-in programs 1 43 for taking in the coded video data and the video pixel data through the taking-in device 1 02 
25 and the video IF 1 01 , respectively; 

full-size encoded videos 151 which have been taken in through the video taking-in device 102 or through the video 
IF 101 and the video encoder 107; 

coded size-reduced videos 153 or reduced-size versions of the encoded videos 151 which have been taken from 
the video encoder 107; 
30 a video information DB (data base) 1 55 comprising information on the videos 151; 

a DBMS (data base management system) 145 for managing the video information DB 155; and 

a DB service program or main program 147 for receiving a service request from one of the clients 20 and offering 

information obtained with a help of the DBMS 155 to the requesting client 20. 

35 [001 3] The jobs of the video DB server 10 are roughly classified into two categories, i.e., a collection and maintenance 
of videos and a retrieval service of the collected videos for the clients 20, which involve a preprocess for preparing a 
coded size-reduced version of a taken-in full-size video and a compile process for compiling a frame of coded video 
data from coded size-reduced versions of retrieved videos from the viewpoint of the invention. So. we discuss the video 
taking-in operation and the retrieval service operation centering on the preprocess and the compile process in the fol- 

40 lowing. 

[0014] If a video to be taken in is a coded one, the coded video is taken in via video taking-in device 102 and stored 
in hard disc as a full size encoded video 151 In order to reduce the size of the taken-in coded video, the taken-in coded 
video is once decoded by the video decoder 103 into a frame of digital video data and then passed to the picture size 
reducing logic 105. 

45 [001 5] If a video to be taken in is in a YIQ format (which is a combination of a luminance (Y) signal and two weighted 
color-difference signals I and Q (in-phase and quadrature)) such as NTSC or PAL, the YIQ format video is converted by 
the video IF 101 into a frame of digital video data, and then passed not only to the picture size reducing logic 105 but 
also to the video encoder 107. The frame of digital video data from the video IF 101 is encoded by the video encoder 
107 into a encoded video, which is then stored in the hard disc 104 as a full size encoded video 151. 

so [0016J The picture size reducing logic 105 reduces the size of the frame of the digital video data into one X * Y-th of 
the frame, i.e., into an X-th in width and a Y-th in height, of the frame. It is assumed that retrieved videos are displayed 
as an array of X-by-Y sized-reduced pictures on the display device 1 31 of the client 20. 

[001 7] The size-reduced video is then encoded by the video encoder 107 and stored in the hard disc 104 as a coded 
size-reduced video 153. 

55 [0018] In the video encoder 107, the input size-reduced video is temporarily stored in a buffer memory 108. If a full 
size frame can be divided into M* * N' macroblocks (M* slices),the size-reduced video stored in the buffer 108 is divided 
into M * N macroblocks as shown in FIG. 4, where M = M7X and N = NYY . If a full size frame comprises 40 * 32 mac- 
roblocks (32 slices) for example and if retrieved videos are to be displayed in an array of 8-by-8 sized-reduced pictures, 
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then each frame of the size-reduced pictures is divided into 5(= 40/8) * 4(= 32/8) macroblocks (4 slices)- Each of the 
macroblock comprises 2 by 2 blocks, each of which comprises 8 by 8 pixels. 

[0019] The video encoder 107 performs a well-known encoding operation on each block of each of the macroblocks. 
Since a variable length coding is performed at the end of the encoding operation, the lengths of blocks are variable, and 

5 accordingly the lengths of slices are also variable. 

[0020] For this reason, each of the coded size-reduced videos 153 is stored in a format as shown in FIG. 5 in this 
specific embodiment. Specifically, assuming that each coded size-reduced video 153 comprises frames {Ff j = 1, 2,...}, 
each frame F, comprises slices {S j k : k =1 , 2,... N} and the length of each slice S J k , is denoted by a fixed-length length 
code L jk (bytes), then the slices of each coded size-reduced video, S 1(1 , S 12 ,..., S, N , S 21 , S 22 ... are stored each 

w headed by a fixed-length length code Lj k . Doing this enables an efficient read out of slice data from each coded size- 
reduced video file 153. Thus, the coded size-reduced videos are prepared for multiple display at a time. 
[0021] If the video DB server 10 receives a video retrieval request with a query expression, then the server 10 passes 
the query expression to the DBMS. In response to a reception of a list of retrieved video IDs, the server 10 compiles a 
frame of coded videos from the coded size-reduced videos identified by the retrieved video IDs. The compiling process 

15 will be described in the following. 

[0022] FIG. 6 is a diagram showing how desired size-reduced pictures are displayed in an array of windows. As shown 
in FIG. 6. the display screen is divided into X*Y windows {W x y : x = 1 , 2....X and y = 1 , 2,... Y}. Each of the rows of win- 
dows comprises N slices (N is the number of slices that constitute a size-reduced video). Assuming that the IDs of the 
coded size-reduced videos to be displayed is given as {Pi: i = 1. 2 V and V ^ X*Y}, the controller 1 13 first assigns 

20 the videos P1 , P2,... .PV to desired respective windows as shown in FIG. 6. The assignment of the videos P1 , P2 ( ... 
.PV may done in any geometric pattern. Then, the controller 113 generates an output (or display) video map according 

to the assignment. FIG. 7 shows an exemplary output video map in case of the videos to be displayed, P1 , P2 PV, 

being arranged as shown in FIG. 6; and FIG. 8 shows another exemplary output video map in case of the videos P1, 
P2 PV being arranged sequentially in windows (1. 1). (2, 1) (X, 1), (1,2) and soon. 

25 [0023] On the other hand, each video Pi comprises frames {F; j = 1, 2, .3 }. After deciding the correspondence 

between each coded size-reduced video Pi and the window (x, yj where the video Pi is to be displayed, the controller 
113 directs the frame compiler logic 109 to begin compiling a frame. 

[0024] It is assumed that the frame rates of the displayed videos P1. P2.... PV are R1. R2, ...RV (frames/sec), the 

frame compiler logic 109 compiles frames {Fk: k = 1 , 2, 3 ] at a rate of R frames/sec. The frame rate of the compiled 

30 frames, R, is preferably in the range from the minimum rate to the maximum rate of the videos P1 , P2....PV. In compiling 
each frame, the frame compiler logic 109 selects suitable frames Fi.j from the videos P1 , P2,... PV in the following man- 
ner. 

[0025] FIG. 9 is a diagram showing a temporal correlation between the compiled frames Fk (k = 1, 2,...) and the 
frames F i(j (j = 1 , 2,...) of a video with a higher frame rate Ri. In FIG. 9, the frame number of the video Pi which is to be 
35 used for the compiled frame Fk is given as the maximum integer j that satisfies the relationship: 

(k-1)/R = (H)/R' ) 



40 Thus, j = [(Ri/R)'(k-1)] + 1 . 

where [A] indicates the maximum integer not greater than A. 

[0026] FIG. 10 is a diagram showing a temporal correlation between the compiled frames Fk (k = 1, 2,...) and the 
frames Fj j (j = 1 , 2,...) of a video with a lower frame rate Ri. As seen from FIG. 10, the frame number k of the compiled 
frame Fk and the frame number j of the video Pi which is to be used for the compiled frame Fk are integers that satisfy 
45 the following relation: 

(l/Rir(j-l)^(1/R)*(k-1)<(l/Ri)1- 



so Solving this with respect to j gives 

(Ri/R)*(k-1) < j ^ (Ri/R)*(k-1) + 1. 



55 Since a frame number j is integer, we again obtain 

j = [(Ri/Rr(k-i)] + i. 
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[0027] Thus, the compiled frames Fk (k = 1, 2,...) can be displayed at any desired rate by selecting from the videos 
P1, P2,... PV frames FiJ (1 = 1, 2,... V) identified by 

j = [(Ri/R)*(k-1)]+ 1 (1) 

5 

for each frame Fk regardless of the frame rates of the videos. 

[0028] FIG. 1 1 is a flow chart showing an exemplary operation of displaying a plurality of videos of various frame rates 
at a time in accordance with the principles of the invention. After creating the output video map, the controller 113 enters 

10 the operation of FIG. 1 1 . In step 202, the compiled frame number k and the number Y of a first video to be included in 
the frame Fk are set to 1. In step 204, set the frame number j of the video Pi according to equation (1). 
[0029] In step 206, a test is made to see if the video is a specific one P^ that is to be displayed from a desired frame 
No. B1 . If so, then a value (B1-1) is added to the frame number j of the video P^ in step 208. It may be desirable to 
display only limited frames as shown in FIG. 12. In FIG. 12, the video PJ 1 = A2 is to be displayed only from frame No. 

is 1 00 through 300, while the other videos A1 , B1 and B2 are to be displayed from the beginning of the videos. In case of 
video A2, a value (100-1) is added to the frame number j in step 208. If there is another video to be displayed from a 
frame other than the first frame, a pair of steps like 206 and 208 is inserted in the "NO" path from step 206 to step 210 
for such video, which is repeated till all of such videos are exhausted. In order to achieve this systematically, it is pref- 
erable to add the fields of the start frame number Fj Bj and the end frame Fj Ei for display to the display video map as 

20 shown in FIG. 1 3 and insert a pair of steps like 206 and 208 for a video with a value in the start frame field of the output 
video map. 

[0030] In step 210, a frame of thus decided frame number is selected for the video Pi. The video ID number i is incre- 
mented in step 212. A test is made in step 214 to see if the videos have been exhausted. If not, the control is returned 
to 204. If the test result is YES in step 214. then the controller 113 directs the frame compiler logic 109 to compile a 

25 frame Fk from the selected frames F-, j F 2 j Fyj in step 216 (the compile operation by the frame compiler logic 109 

will be detailed later). 

[0031] In step 218, a test is made to see if the compiled frame number k is equal to (E1-B1)+1 . where E1 is a last 
frame number to be displayed for the specific video Pl-i (= A2 in this example). If so. the controller 1 13 deletes the ID of 
the video Ph from the output video map in step 220 because there is no need of displaying the video Pl-i any more. In 

30 case of A2 of FIG. 12, if k = (299-100)+1 = 200 in step 218. the video A2 is deleted from the output video map, which 
causes the frame F n 201 and the subsequent frames to be displayed no more. If there is another video to be displayed 
only for the frames from frame number B2 to E2, a pair of steps like 218 and 220 is inserted in the "NO" path from step 
218 to step 222 for such video, which is repeated till all of such videos are exhausted. In order to achieve this system- 
atically, it is preferable to insert a pair of steps Jike 218 and 220 for a video with a value in the end frame field of the 

35 output video map. 

[0032] In step 222. the compiled frame number k is incremented. In step 224 a test is made to see if the multiple video 
display operation is to be continued. This test may be made in various ways,. for example, by comparing the compiled 
frame number, the elapsed time from the beginning of the display operation, or the frame number of specific video with 
a preset value or by checking if a stop instruction has been issued. If not, the controller is passed to step 204. If the test 
40 result is YES in step 224, the controller 1 13 ends the operation. In this way. the video DB server 10 enables a plurality 
of videos of various frame rates to be displayed at a time at any desired speed. 

[0033] FIG. 1 4 is a flow chart showing an exemplary operation in which the frame compiler logic 109 compiles a dis- 
play frame Fk from the selected frames F-i j, F 2 j F V j in response to the direction from the controller 113. In FIG. 13, 

the compiler logic 109 first sets the y coordinate of the current window to 1 in step 401 , sets the slice number sin the 

45 window row for y=1 to 1 in step 402, and sets the x coordinate of the current window to 1 in step 403. 

[0034] In step 404, the compiler logic 109 makes a test to see if any video is associated with a window (x, y) in the 
output video map. If so, the logic 109 output the s-th slice Sj j s of the selected frame Fj j of the video associated with the 
window (x, y). Otherwise, the logic 109 outputs a predetermined code indicative of the absence of video in step 406. 
Referring to FIGs. 6 and 7, we Specifically discuss this matter. If (x, y) = (2, 1) in FIG. 6, since no video is associated 

so with the window in FIG. 7 in step 404, the compiler logic 109 outputs the predetermined code. If (x, y) = (3, 1), since a 
video P1 is associated with the window in FIG. 7 in step 404, the compiler logic 109 outputs the first slice S-, j j of the 
selected frame F 1(J - of the video P1 associated with the window (3, 1). In this case, the frame number j to be output is 
given as: 

55 j = [(R1/R)*(k-1)]+ 1. 



according to equation (1). If k = 1 , then j=:1 . That is, the logic 109 searches a P1 file for the length code of the first slice 
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of the first frame, L, A , and outputs the data of (1 in length which data follows the length code L, A 
[0035] In step 407, the x coordinate is incremented. In step 407, a test is made to see if x > X. If so, the logic 109 
returns to step 404. Otherwise, the logic 109 proceeds to step 409, where the slice number s is incremented. In step 
410, a test is made to see if s > N or if the slices in the window has been exhausted. If so. the y coordinate is incre- 

£ mented in step 411. Otherwise, the control is passed to step 403. In step 412, a test is made to see if y > Y. If not, the 
control is passed to step 402. If y > Y in step 41 1 , then the compiler logic 1 09 ends the operation. In this way, a compiled 
frame including frames from the videos P1, P2....PV are transmitted to the requesting client 20. Thereafter, each time 

the controller 113 executes the step 216 of FIG. 1 1 for the subsequent display frames F2, F3, the compiler logic 109 

executes the operation of FIG. 14. 

w [0036] In response to a reception of the compiled frame Fk, The requesting client 20 decodes, analog-to-digital con- 
verts and NTSC/PAL encodes the received compiled frame Fk with the element 123, 125 and 127 to display an array 
of size-reduced videos identified by the retrieved video IDs. 

[0037] By using the output video map, the size-reduced videos may be arranged in any desired pattern on the window 
array shown in FIG. 6. The sizes of the displayed size-reduced videos has not necessarily to be the same. One or more 
is videos may be doubled or tripled in horizontal and/or vertical dimension. 

[0038] It is also noted that the displayed pictures has not necessarily to be from different videos. Instead, some or all 
of the displayed pictures may be from an identical video. 

[0039] In the above-described embodiment, the coded size-reduced videos 153 are stored with the slice length code 
Lj s as shown in FIG. 5. Alternatively, the slices Sj s of a video Pi may be obtained referring to slice information generated 
20 by the video decoder. 

Embodiment II 

[0040] FIG. 15 is a schematic block diagram showing a video decoder 167 used in a video DB server in accordance 
25 with a second embodiment of the invention. In FIG. 15. the video encoder 167 further comprises a slice information 
(INFO) generator 169 through witch a desired slice S j s of a video Pi can be obtained and a multiplexer (MUX) 171 for 
mutiplexing the coded sized-reduced video Pi data, corresponding slice INFO and stuffing bits as detailed later. 
[0041] FIG. 16 is an example of a slice information table 180 generated by the slice INFO generator 169 at the time 
of encoding the size-reduced video Pi. Each record of the table 169 contains a frame number j 181 , a slice number s 
so 1 83, an offset 185 from the beginning of the Pi file 153, and a slice length Lj s 187. It is noted that the offset for the first 
slice (s=1) of the first frame 0=1) is not zero in value because there are various data such as headers before the first 
slice information. A slice INFO table 169 for a video Pi are preferably stored in association with a corresponding video 
file Pi, for example, by giving the same name to the table but differentiating the file name extension. 
[0042] When a slice Sj s is to be used, the frame compiler logic 109 uses the offset 185 and the slice length 187 asso- 
35 ciated with j and s instead of using the slice length of FIG. 5. 

[0043] In order to enable the full size encoded videos and the coded size-reduced videos to be decoded by a standard 
decoder such as an MPEG (Moving Picture Experts Group) decoder, it is preferable to use such a standard encoders 
for the encoder 167 and the same standard decoder for the decoder 103 and 123 of FIG. 2. 

[0044] For this purpose, the multiplexer (MUX) 1 71 preferably generates a multiplexed data stream 180 as shown in 
40 FIG. 1 7. The stream 180 comprises a series of packs. In this specific example, each pack comprises a video data part 

183 containing 8 frames of coded sized-reduced video data, slice table data part 184 containing slice information for 

the eight frames. Each pack may further includes a padding data part 185 to make the packs constant in length. It is 

assumed that each pack including the padding data part 1 87 causes the length of each pack to be L0 bytes and that 

the lengths of the three parts 183-185 are L1 , L2 and L3, respectively. 
45 [0045] The three parts 183-185 begin with 32-bit packet start codes PSC1, PSC2 and PSC3, respectively. Each 

packet start code comprises a 24-bit fixed value code and an 8-bit ID code for identifying the subsequent data. 

[0046] The length L1 equals 32 bits plus the length of 8 frames of coded sized-reduced video data. 

L2 = (Lpi + Ls!)*N*FN + 3 bytes, 

50 

where Lpi is a length of the position information of a slice, Lsl is a length of the slice length, N is the number of slices 
constituting a coded video, FN is the number of frames in a pack (8 in this example). If Lpi and Lsl are 4 bytes, then L2 
= 64N + 3 bytes. 

55 

L3 = L0 - (L1 + L2) 
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[0047] Thus, the multiplexer 1 71 actually outputs padding data of the length equal to L3 - 3 bytes, is., L0-(L1 +L2 + 3) 
bytes. 

[0048] The controller 1 13 and the frame compiler logic 109 uses the data following the start code PSC1 as 8 frames 
of coded sized-reduced video data and the data following the code PSC2 as the slice table 180 data for the 8 frames 

5 and neglects the data following the code PSC3. 

[0049] If we assume that the multiplexed data stream 1 80 is an MPEG bit stream, each pack is a PMEG system pack, 
each video data part is a video packet. The ID code in the packet start code PSC1 can take a value E0 through EFH (H 
denotes a hexadecimal). The ID codes in the packet start codes PSC2 and PSC3 is preferably set to BFH to deal with 
the slice data part and the padding data part as Private 2 packets defined in the MPEG standard. If the multiplexer 171 

10 is so arranged as to generate a multiplexed bit stream or Pi bit stream that includes the codes necessary as a pack 
header and a system header, it enables the multiplexed coded size-reduced video data stream, which include the slice 
table data for use in frame compiling, to be decoded by an ordinary MPEG decoder. It is also possible to arrange the 
multiplexed coded video bit stream such that the multiplexed coded bit stream is in an AVI (Audio Visual Interleaved) 
format or in a DV (DVCPRO) format. 

15 [0050] A method of obtaining a frame from the Pi bit stream will be described in the following. Assume that a frame 
100 (= j), i.e.,Fj t100t is to be obtained. 

[0051] Since each pack contains 8 frames, the pack number Np of the pack containing the frame j is given as: 

Np = < j/8 > 



where < A>indicates a minimum integer not smaller than A. Thus, the frame compiler logic 109 reads L0 bytes of data 
following L0*(13-1) (= 12*L0) bytes that start with the beginning of the Pi bit stream 180. If we assume LO to be 8192, 
then the logic 109 reads 8192 bytes of data following 106496 (= 12*8192) bytes that start with the beginning of the bit 
25 stream 180. By using slice data, any slice can be read. 

Modification I 

[0052] The above-described embodiments have used a single hard disc 1 04. However, the server 1 - may be provided 
30 with a plurality (Nd) of hard discs 104a, e.g.. 4 discs D1, D2.D3 and D4 as shown in FIG. 18. In this case, the video 
encoder 107 or 1 67 preferably stores the frames of a video Pi data in the following manner (the decimal numerals below 
indicate frame numbers). 

D1: 1 ,5, 9. 13. 17,... 
35 D2: 2,6.10,14,18... 
D3: 3,7,11,15,19,... 
D4: 4, 8, 12, 16. 20. .. 

Since each pack contains eight frames, packs stored in the disc D1 comprises following frames: 

40 

Pack D 1/1: 1. 5, 9. 13, 17, 21, 25, 29 

PackDl/2: 33, 37, 41, 45, 49, 53, 57 

45 Pack D 1/3: 61, 65, 69. 73, 77, 81, 85 



so 

[0053] The packs D1/1 , 2, 3... are stored in the hard disc in order of D1/1 , 2, 3 and so on. 

[0054] The frame compiler logic 109 creates packs D1/1. D2/1 , D3/1 and D4/1 from video data, slice information and 
padding data of the frame data from frames 1 to 32. The packs D1/1 . D2/1 , D3/1 and D4/1 are stored in respective hard 
discs D1. D2. D3 and D4. Packs D1/2. D2/2. D3/2 and D4/2 are created from frames 33 to 64 and stored in respective 
55 discs. 

[0055] A read of a frame is achieved either by specifying a frame number or by specifying a pack number. If a frame 
number j is specified, then the hard disc D# storing the frame j is given as: 
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# = (j mod Nd) 

where # is a disc number, Nd is the number of discs (4 in this case), and (X mod Y) indicates the remainder which 
5 results from the division of j/Nd. The number of the pack containing the specified frame j is given as < j/(Nd*Nf) \ where 
Nf is the number of frames in a pack, and < A) is a minimum integer not smaller than A. 

[0056] Doing this causes the load involved in the access to the hard disc 104 to be distributed in a plurality of hard 
discs. 

10 Modification II 

[0057] The above-described video DB system 1 is in a client-server configuration. However, a system embodying the 
invention may be in a stand-alone configuration. 

[0058] FIG. 19 is a schematic block diagram showing a stand-alone system that incorporates an illustrative embodi- 
es ment of the invention. In FIG. 19, the system 2 comprises: 

the above-described video taking-in device 102; 
the frame compiler logic 109 

a video decoder 323 for decoding the taken-in coded video data into video pixel data and for decoding a compiled 
20 frame of size-reduced videos from the frame compiler logic 109; 

the hard disc 104; 
the picture size reducing logic 105; 

a video encoder 107a for encoding the size-reduced video data from the picture size reducing logic 105 into a 
coded size-reduced video for display, which is then stored in the hard disc 103; 
25 a video output IF 330 which has functions of above-mentioned DAC 125. the NTSC/PAL encoder 127 and a display 

interface; 

a controller 310 for controlling the entire system 2; 

an input user interface 340 for providing the controller 310 with an interface with the user; and 
the input devices 1 33. 

30 

[0059] Since the operation of the system 2 is the same as that of the system 1 except that the system 2 has no need 
of communication facility between the frame compiler logic 109 and the decoder 323, the description of system opera- 
tion will be omitted. 

35 Embodiment III 

[0060] FIG. 20 is a schematic block diagram showing a remote monitoring system 3 that incorporates a third illustra- 
tive embodiment of the invention. In FIG. 20, the remote monitoring system 3 comprises a plurality of remote terminals 
50 for transmitting a coded size-reduced version of video data, a center device 70 for collecting and displaying the 
40 coded size-reduced videos at a time, and transmission paths 60 for transmitting respective coded size-reduced videos. 
[0061 ] Each of the remote terminal 50 comprises: 

a video camera 501 for obtaining video frames; 

a camera interface (IF) 503 having the input thereof connected with the camera 501 ; 
45 a picture size reducing logic 505 for reducing the size of the video pixel data from the camera interface 503 into an 

appropriate size; 

a video encoder 507 for encoding the size-reduced video data from the picture size reducing logic 505 into a coded 
size-reduced video for display; 

a communication interface 509 for transmitting the coded size-reduced video to the center device 70; and 
so a controller 51 1 for controlling the entire terminal 50. 

[0062] The center device 70 comprises: 

communication interfaces 71 for communicating with the remote terminals 50; 
55 a frame compiler 75 for compiling a frame comprising an array of coded videos from desired ones of the coded size- 
reduced videos; 

a video decoder 77 for decoding a frame of coded size-reduced videos from the compiler 75; 
a video output IF 79 for converting the decoded frame for display; 
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a controller 83 for controlling the operation of the video decoder 123; 

a display device 81 connected with the user IF 129 for displaying the frame of size-reduced videos; and 
input devices 82 such as a keyboard, a mouse, etc. 

5 [0063] In operation of each remote terminal 50, a shot video frame is passed to the picture size reducing logic 505 
via the camera IF 503. The picture size reducing logic 505 reduces the size of the video frame into a size specified the 
controller 51 1 . The size-reduced picture is decoded by the video decoder 507 and transmitted to the center device 70 
via corresponding transmission path. 

[0064] In center device, the communication interfaces 71 associated with the remote terminal 50 receive the coded 
10 size-reduced pictures from respective remote terminals and passes the coded size-reduced pictures to the frame com- 
piler logic 73, which compiles a display frame from the coded size-reduced pictures on the basis of the above-described 
output video map. Thereafter, the compiled frame is displayed in the same manner as in the above embodiments. 
[0065] Many widely different embodiments of the present invention may be constructed without departing from the 
spirit and scope of the present invention. It should be understood that the present invention is not limited to the specific 
75 embodiments described in the specification, except as defined in the appended claims. 

Claims 

1 . A method of displaying a plurality of specified ones of pictures in an array of windows provided on a display screen 
20 of a computer based system, the method comprising the steps of: 

reducing in size said pictures into size-reduced pictures with a size of said windows; 
coding said size-reduced pictures into coded size-reduced pictures; 
storing said coded size-reduced pictures in a mass storage device; 
25 in response to a predetermined signal and a list of said specified pictures, compiling a frame comprising coded 

size-reduced pictures for said specified pictures; and 
decoding said compiled frame for display. 

2. A method as defined in claim 1 , wherein said coding step comprises the step of coding said size-reduced pictures 
30 such that said coded size-reduced pictures comprises N slices where N being a positive integer, wherein said stor- 
ing step comprises the step of storing each of said slices with a length code heading said slice, said length code 
indicating a length of said slice, and wherein said compiling step comprises the step of, for each of said coded size- 
reduced pictures, reading N said slices using said length codes beading respective slices. 

35 3. A method as defined in claim 1 , wherein said coding step comprises the steps of: 

coding said size-reduced pictures such that said coded size-reduced pictures comprises N slices where N 
being a positive integer; and 

generating and storing slice information which associates each of said coded size-reduced pictures with a 
40 length and a distance from the beginning of said each coded size-reduced picture, and wherein said compiling 

step comprises the step of, for each of said coded size-reduced pictures, reading N said slices using said slice 
information. 

4. A method as defined in claim 1 , further comprising the step of, in response to said list, generating and storing a 
45 map associating said specified pictures in said list with said windows, and wherein said compiling step comprises 

the step of arranging said coded size-reduced pictures in said array of frames according to said map. 

5. A method as defined in claim 4, wherein said compiling step comprises the step of, in response to an absence of 
a picture for a window, providing a predetermined code to this effect and wherein said decoding step further com- 

50 prises the step of in response to said predetermined code, generating such data as cause nothing to be displayed 

in said window. 

6. A method as defined in claim 1, wherein said pictures are derived from respective videos (or moving pictures), 
wherein said coding step further comprises the step of generating frame rates for said respective videos, each of 

55 said frame rates indicating a number of frames per second, and wherein said compiling step comprises the step of. 
in response to said predetermined signal and a list of specified videos, compiling a series of frames, each of said 
frames comprising coded size-reduced pictures derived from respective said specified videos. 
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7. A method as defined in claim 6, wherein said step of compiling a series of frames comprises the steps of: 

for each of said specified videos and for each of said frames, selecting a coded size-reduced picture which is 
last but not preceding said each frame, on a basis of said frame rates and a frame rate of said series of frames; 
£ and 

compiling said series of frames at any desired constant speed. 

8. A method as defined in claim 7, further comprising the step of setting said frame rate of said series of frames equal 
to one of said frame rates for said respective videos. 

10 

9. A method of displaying a plurality of specified ones of videos (or moving pictures) in an array of windows provided 
on a display screen of a computer based system, the method comprising the steps of: 

reducing in size said videos into size-reduced videos with a size of said windows; 
is coding said size-reduced videos into coded size-reduced videos each comprising N slices; 

generating and storing slice information which associates each of said coded size-reduced videos with a length 
and a distance from the beginning of said each coded size-reduced video; 

multiplexing said coded size-reduced videos and said slice information into a series of packs each comprising 
a video data part and a slice information part, said video data part comprising a first header including a first ID 
20 code and a predetermined number of frames of said each coded size-reduced video, said slice information part 

comprising a second header including a second ID code and slice information for said predetermined number 
of frames; 

in response to a predetermined signal and a list of said specified videos, for each of said coded size-reduced 
videos, reading N said slices from said series of packs by using said slice information to compile a series of 
25 frames, each of said frames comprising coded size-reduced frames derived from respective specified videos; 

and 

decoding said series of compiled frames for display. 

10. A method as defined in claim 9, wherein said multiplexing step includes the step of including, in each of said packs, 
30 a padding data part comprising said second header including said second ID code and a length of padding byte, 

said length being such that said packs become equal in length. 

11. A method as defined in claim 9, further comprising the step of storing said series of packs in a plurality of mass 
storage devices such that said series of packs are uniformly distributed to said mass storage devices. 

35 

12. A system of displaying a plurality of specified ones of pictures in an array of windows provided on a display screen 
of a computer based system, the system comprising: ; 

i 

means for reducing in size said pictures into size-reduced pictures with a size of said windows; 
40 means for coding said size-reduced pictures into coded size-reduced pictures; 

means for storing said coded size-reduced pictures in a mass storage device; 

means, responsive to a predetermined signal and a list of said specified pictures, for compiling a frame com- 
prising coded size-reduced pictures for said specified pictures; and 
means for decoding said compiled frame for display. 

45 

13. A system as defined in claim 12, wherein said coding means comprises means for coding said size-reduced pic- 
tures such that said coded size-reduced pictures comprises N slices where N being a positive integer, wherein said 
storing means comprises means for storing each of said slices with a length code heading said slice, said length 
code indicating a length of said slice, and wherein said compiling means comprises means for, for each of said 

so coded size-reduced pictures, reading N said slices using said length codes heading respective slices. 

14. A system as defined in claim 12, wherein said coding means comprises: 

means for coding said size-reduced pictures such that said coded size-reduced pictures comprises N slices 
55 where N being a positive integer; and 

means for generating and storing slice information which associates each of said coded size-reduced pictures 
with a length and a distance from the beginning of said each coded size-reduced picture, and wherein said 
compiling means comprises means for, for each of said coded size-reduced pictures, reading N said slices 
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using said slice information. 

1 5. A system as defined in claim 1 2, further comprising means, responsive to said fist, for generating and storing a map 
associating said specified pictures in said list with said windows, and wherein said compiling means comprises 

5 means for arranging said coded size-reduced pictures in said array of frames according to said map. 

16. A system as defined in claim 15, wherein said compiling means comprises means, response to an absence of a 
picture for a window, for providing a predetermined code to this effect and wherein said decoding means further 
comprises means response to said predetermined code for generating such data as cause nothing to be displayed 

10 in said window. 

17. A system as defined in claim 12, wherein said pictures are derived from respective videos (or moving pictures), 
wherein said coding means further comprises means for generating frame rates for said respective videos, each of 
said frame rates indicating a number of frames per second, and wherein said compiling means comprises means, 

75 responsive to said predetermined signal and a list of specified videos, for compiling a series of frames, each of said 

frames comprising coded size-reduced pictures derived from respective said specified videos. 

18. A system as defined in claim 1 7, wherein said means for compiling a series of frames comprises: 

20 means, operative for each of said specified videos and for each of said frames, for selecting a coded size- 

reduced picture which is last but not preceding said each frame, on a basis of said frame rates and a frame rate 
of said series of frames; and 

means for compiling said series of frames at any desired constant speed. 

25 19. A system as defined in claim 18, further comprising means for setting said frame rate of said series of frames equal 
to one of said frame rates for said respective videos. 

A system of displaying a plurality of specified ones of videos (or moving pictures) in an array of windows provided 
on a display screen of a computer based system, the system comprising: 

means for reducing in size said videos into size-reduced videos with a size of said windows; 
means for coding said size-reduced videos into coded size-reduced videos each comprising N slices; 
means for generating and storing slice information which associates each of said coded size-reduced videos 
with a length and a distance from the beginning of said each coded size-reduced video; 
means for multiplexing said coded size-reduced videos and said slice information into a series of packs each 
comprising a video data part and a slice information part, said video data part comprising a first header includ- 
ing a first ID code and a predetermined number of frames of said each coded size-reduced video, said slice 
information part comprising a second header including a second ID code and slice information for said prede- 
termined number of frames; \ ! 

means, responsive to a predetermined signal and a list of said specified videos and operative for each of said 
coded size-reduced videos, for reading N said slices from said series of packs by using said slice information 
to compile a series of frames, each of said frames comprising coded size-reduced frames derived from respec- 
tive specified videos; and 

means for decoding said series of compiled frames for display. 

A system as defined in claim 20, wherein said multiplexing means includes means for including, in each of said 
packs, a padding data part comprising said second header including said second ID code and a length of padding 
byte, said length being such that said packs become equal in length. 

so 22. A system as defined in claim 20, further comprising means for storing said series of packs in a plurality of mass 
storage devices such that said series of packs are uniformly distributed to said mass storage devices. 

23. A system as defined in claim 20, wherein said decoding means including means for decoding only data headed by 
said first ID code while neglecting data headed by said second ID code. 

55 

24. A server that enables a client terminal to display a plurality of specified ones of pictures in an array of windows pro- 
vided on a display screen of said client terminal, the server comprising: 
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means for reducing in size said pictures into size-reduced pictures with a size of said windows; 
means for coding said size-reduced pictures into coded size-reduced pictures; 
means for storing said coded size-reduced pictures in a mass storage device; 

means, responsive to a predetermined signal and a list of said specified pictures, for compiling a frame com- 
5 prising coded size-reduced pictures for said specified pictures; and 

means for transmitting said compiled frame to said client terminal. 

25. A video database server that enables a client terminal to display a plurality of retrieved ones of stored videos in an 
array of windows provided on a display screen of the client terminal, the server comprising: 

10 

means for reducing in size said pictures into size-reduced pictures with a size of said windows; 
means for coding said size-reduced pictures into coded size-reduced pictures; 
means for storing said coded size-reduced pictures in a mass storage device; 

means, responsive to a reception of a request from a requesting client terminal, for providing a predetermined 
75 signal and a list of retrieved pictures; 

means, responsive to said predetermined signal and said list of retrieved pictures, for compiling a 1rame com- 
prising coded size-reduced pictures for said retrieved pictures in said list; and 
means for transmitting said compiled frame to said requesting client terminal. 

20 26. A remote monitoring system comprising: 

a plurality of remote terminals; and 

a center device for collecting video data from said remote terminals and display said collected videos in an 
array of windows provided on a display screen, wherein each remote terminal comprises: 
25 a video camera for providing a shot video; 

means for reducing in size said video into size-reduced video with a size of said windows; 
means for coding said size-reduced video into coded size-reduced video; and 

means for communicating said coded size-reduced video to said center device, and wherein said center device 
comprises: 

30 means for receiving said coded size-reduced video from each remote terminal 

means for compiling a frame comprising said received coded size-reduced videos from said remote terminals; 
and 

means for decoding said compiled frame for display. 

35 
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